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SOME FACTORS IN A PROBABILITY SAMPLE SURVEY 
OF A METROPOLITAN COMMUNITY * 


HARRY SHARP AND ALLAN FELDT 
University of Michigan 


Surveys conducted by the University of Michigan’s Detroit Area Study in 1956, 1957, and 
1958 employed an area-probability sample of metropolitan Detroit adults. No substitutions 
were permitted, nor was a specific limit put on the possible number of call-backs. To achieve 
the 2,313 interviews necessary for an 87 per cent response rate, a total of 7,743 calls was 
required. Although three-quarters of the completed interviews were obtained after three calls, 
there is evidence that biases in the sample would have resulted had the interviewing stopped 
after two call-backs. Available data indicate that the amount of dissimilarity between respond- 
ents and nonrespondents is probably not large enough to influence the representativeness of the 
completed sample, given the response rate of these surveys. 


EW students of survey research would 
kK challenge the claim that sampling tech- 

niques based on _ strict probability 
models afford far more reliable and valid 
results than do other types of sampling.? 
To be sure, for some purposes and under 
some conditions quota samples and ‘mod- 


* A condensed version of this paper was presented 
at the annual meeting of the American Sociological 
Society, 1958. The research was made possible by a 
grant from the Faculty Research Fund of the Horace 
H. Rackham School of Graduate Studies of the Uni- 
versity of Michigan. We wish to express our appre- 
ciation to the Fund. We also wish to thank Morris 
Axelrod, Don Cahalan, and Charles Cannell for their 
suggestions. 

1Qne exception to this claim may be found in 
Francis E. Lowe and Thomas E. McCormick, “Some 
Survey Sampling Biases,” Public Opinion Quarterly, 
19 (Fall, 1955), pp. 303-315. These authors con- 
clude: “. . . it remains to be demonstrated that, 
for ordinary polling purposes, a random sample, in 
the usual statistical sense, can be drawn with con- 
fidence from human populations under a democratic 
way of life, with prevailing field methods.” (p. 315) 
It has been demonstrated repeatedly that an ade- 
quately designed probability sample is capable of 
producing units which are representative, within 
known confidence limits, of a given research uni- 
verse. See, e.g., C. A. Moser and A. Stuart, “An 
Experimental Study of Quota Sampling,” Journal 
of the Royal Statistical Society, A, 116 (1953), pp. 
349-405; Leslie Kish, “A Procedure for Objective 
Respondent Selection Within the Household,” Jour- 
nal of the American Statistical Association, 44 (Sep- 
tember, 1949), pp. 380-387; Bernard R. Berelson, 
Paul F. Lazarsfeld, and William N. McPhee, Voting, 
Chicago: University of Chicago Press, 1954, Appen- 
dix C; A Social Profile of Detroit: 1956, Ann Arbor: 
The Detroit Area Study, 1957, Appendix.A. See also 
Morris H. Hansen, William N. Hurwitz, and William 
G. Madow, Sample Survey Methods and Theory, 
New York: Wiley, 1953, Volume I, Chapter 1. 


ified” probability samples may actually be 
preferable to more rigorously designed re- 
search; but even in these situations most 
survey researchers would prefer to base their 
analyses on probability models, if costs and 
time were not considerations.” 

A basic charge leveled against quota sam- 
pling is that ultimately the interviewer is 
also the sampler. The interviewer makes the 
final decision as to which of the many units 
in the research universe will fall into the 
sample. He may choose to interview those 
persons within his quota who are most willing 
to be respondents and are easiest to locate. 
It does not necessarily follow that these 
“volunteers” are a representative sample of 
all possible units with similar quota char- 
acteristics.2 They could, in truth, be repre- 
sentative, but there is no statistically accept- 


2W. R. Simmons and others recently made this 
point during a panel discussion of the question, “Is 
Probability Sampling Overdone?” at the 1958 meet- 
ing of the American Association for Public Opinion 
Research. Simmons’ conclusion was that true proba- 
bility sampling is underdone. “Proceedings of the 
Thirteenth Conference,” Public Opinion Quarterly, 
22 (Summer, 1958), p. 208. See also C. A. Moser, 
Survey Methods in Social Investigation, New York: 
Macmillan, 1958, pp. 106-108. 

3 Evidence from an experimental study in England 
indicates that on many items quota samples do not 
differ greatly from random samples. On some very 
crucial items, however, the quota sample is drasti- 
cally in error. In addition to large biases, the vari- 
ance of the quota samples in this research were from 
150 per cent to about 400 per cent larger than the 
random sample variance. Thus, in terms of standard 
error per dollar the quota sample actually may be 
more expensive than a probability sample. Moser 
and Stuart, op. cit. 
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able method by which confidence limits for 
a nonprobability sample can be ascertained. 

In a strict probability sample all units in 
a defined universe are given an equal or 
known nonzero chance of being selected. The 
interviewer has no opportunity whatsoever 
to determine the specific units which he must 
contact. Unfortunately, the problem of ob- 
taining a representative sample with known 
confidence intervals is not solved simply by 
drawing a probability sample. This would be 
the case only if (1) all units which fell into 
the original sample yielded a completed 
interview, or if (2) nonrespondents differed 
from respondents solely in their failure to 
grant an interview. The dilemma for prob- 
ability sampling on most human populations 
is that neither of these conditions holds. 

Survey researchers have grappled with the 
nonresponse problem for many years.* A 
considerable body of valuable data is now 
available which can help in making estimates 
of costs and in anticipating and correcting 
biases and weak points in the interviewing 
stage of a survey. Few studies of response 
rate which are readily accessible, however, 
have centered on a large metropolitan area 
in the United States. Almost all of our 
information about this important aspect of 
survey research is based upon studies of 
relatively small communities or of the entire 
nation. To an ever increasing extent the 
metropolitan community is of central interest 
to the social sciences. In the present paper 
an attempt is made to describe in some detail 
the factors which are involved in carrying 
through a large-scale survey in a metropol- 
itan center with a population of over three 
and one-half million persons. 


THE DATA 


Each year since 1952 The University of 
Michigan’s Detroit Area Study *® has con- 
ducted an annual survey of the adult popula- 
tion of the Detroit metropolitan community. 


4Nonresponse is also an important concern of 
compulsory censuses. See Morris H. Hansen, William 
N. Hurwitz, and Leon Pritzker, “The Accuracy of 
Census Results,” American Sociological Review, 18 
(August, 1953), pp. 416-423. 

5 The Detroit Area Study is associated with the 
Department of Sociology and the Survey Research 
Center of the Institute for Social Research at the 
University of Michigan. 
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This discussion focuses on the surveys of 
1956, 1957, and 1958. The objectives of these 
surveys required a sample of the adults of 
greater Detroit which would be representa- 
tive, within known confidence limits, of all 
noninstitutionalized adults in the commu- 
nity.® 

Area-probability samples were constructed 
for this research which allowed the assign- 
ment of specific addresses at which interviews 
were to be obtained.’ No substitutions of 
any kind were permitted. The interviewer’s 
first task at a designated dwelling unit was 
to select from all adult residents one person 
as the respondent. The selection was pre- 
determined for the interviewer, following a 
method described in the literature.’ Again, 
no substitutions were permissible at this stage 
of the sample design. 

All three surveys used an interview sched- 
ule which required an average of one hour 
and ten minutes to administer. Standard 
open-ended and closed-answer questions were 
employed in about equal numbers. The topic 
of the research, of course, varied each year.® 

In each of these surveys detailed informa- 
tion was gathered about the interviewing 
stage. The situation surrounding every con- 
tact at every sample address was noted fully 
by the interviewer. The analyses of these 
data should present some valuable insights 


6 The comparability of Detroit Area Study sample 
distributions with those of the Census attest to the 
validity of the sample. See A Social Profile of 
Detroit: 1956, loc. cit. 

7 Detroit Area Study samples were clustered and 
stratified to meet the needs of the research projects. 
Detailed descriptions of the specific probability 
models used in each survey are available from the 
Detroit Area Study. The sample universe, which is 
that part of the greater Detroit area that is tracted 
by the Census, includes about 87 per cent of the 
population of the three-county Detroit Standard 
Metropolitan Area. The samples were designed by 
the Sampling Section of the Survey Research Center. 

8 Kish, op. cit. 

9In 1955-56 the facilities of the Detroit Area 
Study were used by Robert C. Angell for research 
in normative integration, and by Robert Weiss and 
Robert Kahn for a study of the sociology of work. 
Daniel Katz and Samuel Eldersveld investigated 
political behavior in 1956-57. Gerhard Lenski’s re- 
search on the religious life of greater Detroit took 
place in 1957-58. Interview schedules used in these 
surveys are available from the Detroit Area Study. 


No evidence is available that the content of a 


specific study influenced that survey’s response rate, 
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TABLE 1. DISTRIBUTION OF CALLS BY NUMBER AND RESULT AT EAacH SAMPLE ADDRESS 


Percentage Distribution by Number of Calls 


Mean No. 
Result of Call Five or Total of Calls 
One Two Three Four More Sample 
Completed interview 28 36 28 31 25 87 2.6 
Refusal or “too busy” 20 16 21 15 18 8 4.6 
Respondent not home 20 14 15 16 13 3 6.4 
No one home 30 32 34 37 43 1 7.0 
Respondent ill, senile 1 1 1 | 1 1 3.2 
Language problem 1 1 1 * * 7 3.7 
Total 100 100 100 100 100 100 2.9 
Number of cases 2,646 1,888 1,164 767 1,278 2,651 7,743 
_* Less than 0.5 per cent. 
into the processes by which metropolitan RESPONSES AND NONRESPONSES 


samples based on probability models are 
constructed ‘‘call-by-call.” As much informa- 
tion as possible was collected on the char- 
acteristics of nonrespondents. These data are 
presented in the final section of the paper. 
For convenience, the combined samples are 
discussed below as though they constituted 
a single survey. 

A combination of budgetary considerations 
and estimates of tolerable confidence limits 
yielded 2,651 addresses in the original sam- 
ple which contained eligible respondents. It 
was determined that a sample of this size 
should produce the desired number of com- 
pleted interviews. This decision was pred- 
icated on a final response rate of 85 per cent. 

No cross-sectional metropolitan survey, of 
course, can plan on a 100 per cent response 
rate. Even if every designated respondent 
were willing to grant an interview, the cost 
involved in reaching the total sample would 
be prohibitive. In general, experience has 
shown that, first, per unit interviewing costs 
in time and money rise dramatically after 
a response rate of about 75 per cent is 
reached, and, second, the distortion in the 
completed sample due to nonresponse does 
not seem to be excessive when the proportion 
of nonresponse is not more than 15 to 18 
per cent of all possible interviews. A bal- 
ancing of these considerations with the desire 
to achieve perfection in response led to the 
decision to settle for 85 per cent completion. 
Nonetheless, any specific response rate figure 
is unavoidably arbitrary. 


The 1956-58 Detroit Area Study surveys 
actually produced 2,313 interviews for a 
final response rate of 87 per cent (see Table 
1). Eighty-seven out of every 100 adults in 
the sample were interviewed only with the 
help of an intensive campaign in which the 
interviewer himself was perhaps the most 
crucial figure.1° 

All sample addresses were sent letters 
(first-class mail) which heralded the arrival 
of an interviewer, explained the purpose of 
the survey, and stressed the fact that no 
product or service was being sold. The inter- 
viewers were also aided by several newspaper 
articles which described the research and 
explained the necessity of talking with a 
certain individual in a certain family. The 
actual value of these devices in reducing the 
nonresponse rate is debatable. There is some 
evidence that their primary effect is that 
of solidifying the incipient negative or posi- 
tive attitudes toward an interview held by 
designated respondents. Most interviewers 
encourage the use of letters and advance 
publicity, however, because “they serve as 
an excellent introduction and ease the initial 
contact with many respondents.” 11 


10 For a comprehensive discussion of the role 
played by the interviewer in the efforts to achieve 
a low nonresponse rate, see Robert L. Kahn and 
Charles F. Cannell, The Dynamics of Interviewing, 
New York: Wiley, 1957, pp. 65-91. 

11 Charles F. Cannell, “Factors Affecting the Re- 
fusal Rate in Interviewing,” a paper presented at 
the 1949 meeting of the American Statistical Asso- 
ciation, p. 7 (mimeographed). 
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Detroit Area Study interviewers were 
highly trained in that extremely important 
phase of survey research which occurs when 
the interviewer first meets the respondent. 
A discussion of the techniques of probability 
sampling also is a basic part of interviewer 
training, for the interviewer often must con- 
vince the respondent of the necessity of 
interviewing him, rather than his wife (who 
“really knows much more about this than 
I do’’) or his neighbor (who “‘just sits around 
all day anyway’’). 

In the modern heterogeneous metropolitan 
community a small number of non-English 
speaking respondents inevitably are included 
-in samples. The Detroit Area Study was able 
to handle the majority of such persons by 
employing a _ Polish-speaking interviewer, 
who translated the schedule and interviewed 
in this language. One per cent of our total 
sample addresses resulted in an interview 
rather than a nonresponse because of this 
provision. 

Thirteen per cent of the eligible adults in 
the sample did not grant an interview. The 
majority of the nonrespondents, but only 
eight per cent of the total sample, was 
classified as refusals.1* In survey research of 
this type “refusals” vary from the rare person 
who is extremely discourteous and seems to 
gain some psychological satisfaction from 
slamming the door in an interviewer’s face 
to the timid and retiring housewife who 
personally would be happy to grant an inter- 
view but has been forbidden to do so by 
her husband. 

At only four per cent of the total sample 
addresses was it impossible to contact the 
designated respondent.!? The incidence of 
not-at-homes was kept to a minimum by 
interviewer calls at any hour of the day or 
night which would suit the convenience of 
the respondent. Also, interviews were occa- 
sionally made at places of work or recreation. 

Around one per cent of the adult residents 
of any large community cannot be inter- 
viewed because of physical or mental defi- 


12 This refusal rate is in line with that found in 
similar surveys reported by Frederick F. Stephan 
and Philip J. McCarthy, Sampling Opinions, New 
York: Wiley, 1958, p. 262. 

13 Stephan and McCarthy (Jbid., pp. 241-242) 
indicate that in most surveys it may be expected 
that the noncontacted group will run between 3 and 
8 per cent of the total sample. 
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ciencies. Short of excluding them from the 
sample by definition, clearly a questionable 


procedure, no amount of effort or money can 
reduce the number of such persons. 


THE PROBLEM OF CALL-BACKS 


By far the greatest contribution to a suc- 
cessful survey is the willingness to make 
repeated calls at dwellings which do not 
yield interviews on the first call. No specific 
limit to the number of calls per dwelling was 
established for the Detroit Area Study re- 
search. Interviewers continued their efforts 
until one of four contingencies occurred: (1) 
an interview was granted, (2) a final and 
definite refusal resulted, (3) the designated 
respondent was judged physically or men- 
tally incapable of granting an interview, or 
(4) a satisfactory response rate was reached. 

Call-backs are an expensive element in 
survey research.!* Two basic techniques have 
been developed in an attempt to reduce or 
eliminate the need for repeated calls with 
probability samples. The first device involves 
the selection of a subsample of that portion 
of the original sample that cannot be reached 
on the first call. Intensive effort is then made 
to obtain complete returns from the sub- 
sample. This scheme includes appropriate 
weighting of the original sample and the 
subsample so as to achieve an unbiased re- 
sult at lowest cost.1> In the second technique 
interviewers call on each sample address only 
once. This visit is made at a randomly 
designated time. Each respondent supplies 
information about his presence or absence 
from home during a randomly selected time 
on each day of the week preceding the single 
interviewer contact. Weights are assigned on 
the basis of probabilities of being interviewed 


14 J. Durbin, however, claims that while repeated 
contacts necessarily raise the total cost of a survey, 
the average cost per interview is not increased 
greatly by call-backs. “Non-Response and Call- 
Backs in Surveys,” Bulletin of the International 
Statistical Institute, 34 (1954), pp. 3-17. 

15 See Morris H. Hansen and William N. Hur- 
witz, “The Problem of Non-Response in Sample 
Surveys,” Journal of the American Statistical Asso- 
ciation, 41 (December, 1946), pp. 517-524. Although 
this article specifically refers to first contacts based 
on mail questionnaires, the principles discussed here 
are applicable to standard personal interview sur- 
veys. 
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on the first call. The weighted sample is then 
used in all further analysis.1¢ 

Considerable controversy exists concerning 
the reliability of each of these methods, their 
administrative efficiency, the problems they 
introduce in analysis, the degree to which 
they require an accurate and difficult to 
obtain foreknowledge of interviewing costs, 
and the degree to which over-all costs are 
actually reduced through their use.’’ Until 
these techniques are further refined or new 
methods for eliminating call-backs are de- 
vised, most survey researchers would prob- 
ably prefer to lower the nonresponse rate by 
repeated, although costly, interviewer con- 
tacts. 


NUMBER OF CALLS FOR RESPONDENTS AND 
NONRESPONDENTS 


The Detroit area research necessitated an 
over-all average of almost three individual 
contacts at each sample address to achieve 
the desired number of interviews. The mean 
number of calls, moreover, shows great var- 
iation according to the final disposition of 
the address (see Table 1). It is perhaps 
paradoxical that completed interviews evi- 
dence by far the lowest mean number of 
calls. The average interview was obtained 
after only 2.6 calls; interviewers found it 
necessary to call on the average nonrespond- 
ent almost twice as frequently. 

Not-at-homes have a substantially higher 
mean number of calls than refusals. Six or 
seven contacts were made, on the average, 
in an effort to reach those adults who are 
at home only to sleep or who may be on 
vacation or, possibly, who never answer the 
door. Our experience with this group of 
unreachables indicates that for all practical 


16 See Alfred Politz and Willard Simmons, “An 
Attempt to Get the ‘Not-at-Homes’ into the Sample 
Without Call-Backs,” Journal of the American Sta- 
tistical Association, 44 (March, 1949), pp. 9-31. 
More recently, Simmons has modified this technique 
so that two or more calls are required; weights are 
applied to completed interviews which are appro- 
priate for the number of calls required to obtain a 
given interview. See Simmons, “A Plan to Account 
for ‘Not-at-Homes’ by Combining Weighting and 
Call-backs,” The Journal of Marketing, 19 (July, 
1954), pp. 42-53. 7 

17 Reviews and evaluations of each technique are 
found in Stephan and McCarthy, op. cit., pp. 255- 
261; and Moser, op. cit., pp. 135-139, 
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purposes they are almost impossible to con- 
tact under any conditions. Obviously, “in- 
creasing the number of calls is not enough, 
by itself, to do away with [sample] mor- 
tality.” 48 

The average refusal was not so designated 
until 4.6 calls were made at his home. Only 
the most adamant nonrespondents were clas- 
sified as refusals on the basis of a single 
display of uncooperativeness. In these very 
rare cases it did not appear physically safe 
for the interviewer to attempt to complete 
an interview. 

Most adults who refused on the first con- 
tact were sent a second letter which again 
explained the study and pleaded for coopera- 
tion. Approximately one-half of these persons 
later granted an interview when contacted 
by a different interviewer. The likelihood of 
a response or a nonresponse resulting from a 
given call is understandably influenced by 
the previous calls that may have been made 
at that address. Well-trained interviewers 
record all information produced by earlier 
unproductive calls that may be helpful in 
winning an eventual interview. 

The chances of a successful first call, at 
least in metropolitan Detroit, are no better 
than three out of ten (see Table 1),!® and 
are about equal to the chances of finding 
no one at home in the dwelling. In two out 
of ten first contacts the designated respond- 
ent either straightforwardly refuses or states 
that he is too busy at the moment to grant 
an interview. When the latter claim is made, 
the interviewers attempt to obtain an ap- 
pointment when an interview will be granted. 

The correct respondent is not at home in 
20 per cent of the first calls, although some- 
one is in the dwelling to provide information 
upon which intelligent call-backs can be 
planned. Efficient interviewers can often use 
this person, who is usually the spouse of the 
designated respondent, as a liaison between 


18 Stephan and McCarthy, op. cit., p. 239. 

19 Researchers in Elmira, New York, and Madi- 
son, Wisconsin, also report a ratio of approximately 
three interviews for every ten sample addresses con- 
tacted for the first time. See Robert Williams, 
“Probability Sampling in the Field: A Case History,” 
Public Opinion Quarterly, 14 (Summer 1950), pp. 
316-330; and Lowe and McCormick, op. cit. An 
English study, however, was able to achieve a four 
in ten ratio for successful first calls; Moser, op. cit., 
p. 135. 
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TABLE 2. DISTRIBUTION OF CALLS FOR COMPLETED INTERVIEWS BY SELECTED DEMOGRAPHIC CHARACTERISTICS 
OF RESPONDENTS 


Percentage Distribution by Number of Calls 


Demographic Mean No. 
Characteristics Number Four or of Calls 
of Cases One Two Three More Total 

All Respondents 2,313 32 30 14 24 100 2.6 
Employment Status 

Employed 1,357 21 31 18 30 100 0 

Not employed 952 48 28 9 15 100 2.1 
Age 

21-29 years 508 34 28 14 24 100 245 

30-39 years 634 28 33 14 25 100 Z.f 

40-49 years 455 29 34 15 22 100 2.6 

50-64 years 502 32 25 16 27 100 2.8 

65 years or older 212 47 25 11 17 100 2.2 
Marital Status 

Never married 186 28 21 14 37 100 cs) 

Divorced, separated 105 26 24 16 34 100 6 

Widowed 163 45 23 11 21 100 2.3 

Married, no. children 

present 864 31 26 16 27 100 2.8 

Married, children present 990 33 36 13 18 100 2.3 
Sex 

Male 1,088 25 27 20 28 100 2.8 

Female 1,225 38 30 12 20 100 2.4 
Relationship to Head of 

Household 

Head 1,176 26 29 17 28 100 2.9 

Wife 910 40 31 11 18 100 Zed 

Other relative 189 33 31 14 22 100 2.6 


the study and the desired individual. If the 
interviewer can impress the value of the re- 
search upon the person first contacted, his 
task in completing the interview will be much 
simpler. As a result of information acquired 
from the first call, the likelihood of com- 
pleting an interview is somewhat higher on 
the second contact than on the first. The 
proportions of refusals and respondents not 
at home decline correspondingly between the 
first and second calls. Easily contacted re- 
spondents, however, are relatively quickly 
removed from the uncompleted sample; the 
proportion of responses drops back to the 
first call level on third and later contacts.° 

The number of dwellings at which no one 
can be found at home becomes a progres- 
sively larger segment of the noninterviewed 
sample with each succeeding call. The dis- 
covery of data about these persons which 


20In smaller communities the proportion of re- 
sponses continues to increase at least through the 
third call. Williams, op. cit.; Lowe and McCormick, 
op. cit. 


would aid in making call-backs on a more 
informed basis is very difficult. Although 
neighbors are frequently contacted in an 
effort to ascertain something about their 
daily routines, these are usually unknown 
to the neighbors themselves. Thus, fully 43 
per cent of the calls made after four contacts 
are at dwellings in which no one is at home. 

The number of calls required to reach a 
satisfactory response rate with the Detroit 
Area Study’s sample appears to be consid- 
erably larger than those reported for other 
surveys.*? As noted above, most of the de- 
tailed information now available on this 
aspect of survey research is derived from 
studies of non-metropolitan areas. Interview- 
ing mobile metropolitan residents, some of 
whom live in highly inaccessible apartment 
buildings, is a much more difficult task than 
is similar research in smaller communities 
or rural areas. As the first line of Table 2 
indicates, if Detroit Area Study researchers 


21 Stephan and McCarthy, op. cit., p. 242; Lowe 
and McCormick, op. cit. 
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had stopped with just two calls, hardly more 
than six out of every ten eventual respond- 
ents would have been reached. Even three 
calls would have resulted in a response rate 
of but 66 per cent, and only three-quarters 
of the final number of completed interviews 
would have been accumulated.” 

The fact that some respondents can be 
easily reached with one call while others 
require eight or nine contacts before granting 
an interview would not influence a sample 
based on only one or two contacts if these 
groups of persons were alike on every char- 
acteristic except accessibility. That they are 
quite dissimilar in many respects is seen 
in Tables 2, 3, and 4, where selected char- 
acteristics of respondents are related to the 
number of calls necessary to complete an 
interview. 

Demographic Characteristics and Call- 
Backs. Adults who do not work outside the 
home are quite easily reached in a fixed- 
address sample survey. Almost one-half of 
all unemployed respondents grant interviews 
on the first contact (see Table 2). Only 
one-fifth of the employed respondents are 
interviewed without call-backs.?? Corre- 
spondingly, twice as large a proportion of 
workers are interviewed after four or more 
calls (30 per cent) as of nonworkers (15 
per cent). 

Age bears no constant relationship to 
accessibility for respondents who are less 
than 65 years old. Very young adults are 
about as easily contacted as are middle- or 
late middle-aged respondents. Beyond age 
65, however, the proportion of interviews 
obtained on the first call rises significantly. 
The rate of retirement is high in this age 
group and the general pace of life has slowed. 
Older respondents, forced to spend many 


22 Again, comparison of these figures with com- 
parable data from Elmira and Madison emphasizes 
the greater effort required when interviewing in 
metropolitan areas. Williams, Joc. cit.; Lowe and 
McCormick, op. cit. See also, H. F. Lydall, British 
Incomes and Savings, Oxford: Basil Blackwell, 1955, 
p. 4; and Cannell, of. cit. 

23 All specific statements of percentage point dif- 
ferences and differences in means which are found 
in the text are statistically significant at the 5 per 
cent level or better. In computing tests of signifi- 
cance allowance was made for the effects of cluster- 
ing in the sample. The general conclusions of this 
paper are also based on statistically significant 
differences. 
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hours in their homes, are quite easily con- 
tacted and no doubt welcome the diversion 
offered by the interview.** 

Unmarried and divorced or separated 
adults would be significantly under-repre- 
sented in a study based on even three inter- 
view attempts. These persons have little to 
keep them at home where they can be con- 
veniently reached by interviewers. On the 
other hand, widowed adults would be 
weighted too heavily in the limited call-back 
survey. A very large proportion of all wid- 
owed respondents (45 per cent) grant inter- 
views on the first contact. The comparative 
ease with which the widowed are interviewed 
is consistent with the tendency for older 
persons, many of whom are not in the labor 
force, to be highly accessible. 

Whether or not married respondents are 
easily contacted is related to the presence 
or absence of children in the house. If there 
are children the chances are quite small that 
both spouses are working. Children act as 
anchors which keep wives, and probably 
husbands, where the interviewer can reach 
them relatively quickly. Married respondents 
with no children are only slightly less diffi- 
cult to contact than are unmarried adults. 

Men, most of whom are in the labor force 
and unavailable during the day, are much 
more difficult to contact than are women. 
Almost one-half of the male respondents in 
our sample required at least three calls. 
Only one-third of the females were equally 
difficult to interview. The implications of 
this well-documented finding are obvious for 
limited call-back attitudinal studies in which 
sex is an important variable. 

The relationship of the respondent to the 
head of the household is also an important 
factor related to his accessibility. Wives are 
comparatively easy to reach while the head 
himself is noticeably more difficult to 
interview. 

Our data do not reveal differences in acces- 
sibility associated with race, residence in the 
suburbs or the central city, or the place of 
birth of respondents. 

Socio-Economic Characteristics and Call- 


24 Stephan and McCarthy (op. cit., pp. 243-244) 
cite several earlier studies of call-backs in survey 
research which reach similar conclusions as to the 
relationship of accessibility to age and to other 
variables discussed in this section of the paper. 
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TABLE 3. DISTRIBUTION OF CALLS FOR COMPLETED INTERVIEW BY SELECTED SociI0-ECONOMIC 
CHARACTERISTICS OF RESPONDENTS 
Percentage Distribution by Number of Calls 
Socio-Economic Number - Mean No. 
Characteristics of Cases Four or of Calls 
One Two Three More Total 
Annual Family Income 
Less than $3,000 264 49 21 10 20 100 2508 
$3 ,000-$4,999 516 34 31 14 21 100 2.5 
$5,000-$6 ,999 582 31 31 14 24 100 2.6 
$7,000—-$9,999 505 28 29 14 29 100 2.8 
$10,000 or more 368 25 31 20 24 100 2.8 
Median income bee $5,500 $6,250 $6,600 $6,450 $6,100 
Occupation of Persons Now 
Working 
Operatives, service work- 
ers, laborers 529 24 28 18 30 100 2.9 
Craftsmen, foremen 246 21 31 20 28 100 2.9 
Clerical, sales workers 291 18 34 17 31 100 3.0 
Professionals, managers, 
proprietors 291 20 32 18 30 100 3.1 
Education 
6 years or less 259 33 29 12 26 100 2.7 
7-8 years 404 34 28 13 25 100 2.6 
9-11 years 580 32 30 15 23 100 2.6 
12 years 679 32 29 16 23 100 2.6 
Some college 387 30 32 15 23 100 2.6 
Mean number of years 10.2 10.4 10.6 10.2 10.3 
Subjective Social Class? 
Working class 431 35 30 15 20 100 235 
Middle class 20 32 32 18 18 100 2.5 


* Data from 1958 survey. 


Backs. Socio-economic status is perhaps the 
primary ‘‘independent” variable in most 
studies of social organization. Definite evi- 
dence is shown in Table 3 that the number 
of calls a researcher is willing to make influ- 
ences the socio-economic composition of his 
completed sample. 

Family income and the accessibility of 
family members are inversely related. The 
chances of obtaining an interview on the first 
call approach one in two for respondents 
whose family income is less than 3,000 dol- 
lars annually; they drop to one in four if 
the respondent’s family income is 10,000 dol- 
lars or more. The annual median family 
income in greater Detroit between 1955 and 
1957 was 6,100 dollars. A median based on 
first calls only would have resulted in a 
figure fully 600 dollars below the over-all 
median.?° 


25 These findings tend to substantiate Parten’s 
claim of the greater accessibility of lower income 
respondents. Mildred Parten, Surveys, Polls, and 
Samples, New York: Harper, 1950, pp. 413-414. 


The relationship between income and ac- 
cessibility is probably a result of differential 
employment rates among families of varying 
incomes. The more adults in the family who 
are working, the less likely is anyone to be 
home at any given time to grant an interview. 
Also, family income is quite low for older 
households with no wage earners and in- 
creases with the number of workers in a 
family who contribute to it. 

Notwithstanding the relationship between 
family income and number of calls, addi- 
tional measures of socio-economic status, 
such as education, occupation, and subjec- 
tive social class placement, are not related 
to accessibility of respondents, at least in 
the Detroit metropolitan area.?® This lack of 
relationship reinforces the view that upper- 


26 Nor were occupation and accessibility related 
in Madison. Lowe and McCormick, loc. cit. A slight 
tendency was noted in Elmira, however, for educa- 
tion and accessibility to vary inversely. Philip J. 
McCarthy, “Sampling Procedure for the 1948 Voting 
Study,” p. 37, (mimeographed). 
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TABLE 4. MEAN NUMBER OF CALLS FOR COMPLETED INTERVIEWS BY EMPLOYMENT STATUS AND 
AGE AND SEX, MARITAL STATUS, AND FAMILY INCOME 


Employed Not Employed 
Age and Sex, Marital Status, Mean No. Mean No. 
and Family Income No. of Cases of Calls of Calls No. of Cases 
Age and Sex 
21-39 years: men 488 3.0 2.1 38 
women 213 3.1 238 398 
40-59 years: men 354 2.9 1.9 30 
women 171 3.0 2,3 256 
60 years or older: men 108 3.0 2.4 68 
women 22 3.0 1.9 158 
Marital Status 
Never married 146 3.5 25 37 
Divorced, separated 67 3.6 Lek 37 
Widowed 55 2.9 2.0 108 
Married, no children present 546 3.1 23 318 
Married, children present 541 2.6 2.0 449 
Annual Family Income 
Less than $3,000 76 3.0 z.0 185 
$3 ,OOO—$4,999 311 2.8 2.0 205 
$5,000-$6,999 346 2.9 Lad 236 
$7,000 or more 589 3.1 Zul 281 


status persons, as such, are not necessarily 
less accessible than are lower-status adults. 
The basic consideration in accessibility is 
whether or not the designated respondent 
is in the labor force. First calls result in 
interviews in a disproportionate number of 
very low-income families because these units 
are apt to have no working members and 
are extremely unlikely to have multiple wage 
earners.7? 

The degree to which accessibility is a 
function of employment status is emphasized 
by the data presented in Table 4. Clearly, 
when employment status is held constant 
the relationship of age and sex to accessibility 
disappears. If older women are working, for 
example, they are as difficult to interview as 
are young men. The tendency for adults with 
children in the home to be more easily con- 
tacted than others, although considerably re- 
duced in effect, holds whether or not the 
parent is working. Moreover, among all em- 
ployed respondents, those persons who are 
single, divorced, or separated continue to 
require more interviewer calls than do mar- 
ried adults or widows. Differences in family 
income which are associated with accessi- 
bility disappear completely when the influ- 


27“Family Income in Greater Detroit: 1951- 
1957,” The Detroit Area Study, 1958 (mimeo- 
graphed). 


ence of employment status on income is 
removed. 

The traditional characteristics upon which 
most quota samples are based are age and 
sex. The relationship of these variables to ac- 
cessibility appears to be primarily a result of 
differential employment rates. Quota sam- 
plers should consider the possibility of adding 
an additional restriction, employment status, 
in the design of their research, particularly 
when they are surveying communities in 
which many women are employed, or the 
number of unemployed and retired is large. 


DIFFERENTIAL RESPONSE RATES 


An over-all response rate of 87 per cent 
was cited above for the 1956-58 Detroit 
Area Study surveys. The response rates for 
specific population types in the sample, how- 
ever, do not necessarily match those for the 
total sample, and data for certain segments 
of the population may be seriously biased 
by differential response rates. In fact, con- 
siderable variation exists among some sub- 
groups in the degree to which our interview- 
ers were successful in their tasks (see 
Table 5). 

Ninety-one per cent of all adults between 
the ages of 21 and 34 years in the sam- 
ple granted an interview. Only 78 per 
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TABLE 5. RESPONDENTS AND NONRESPONDENTS BY SELECTED CHARACTERISTICS 
Percentage Distribution 
Nonrespondents 
Selected Re- Not-at- No. of 
Characteristics spondents | Refusals Home Others Total Cases 
Age 
21-34 years 91 4 4 1 100 918 
35-59 years 87 8 4 1 100 1,282 
60 years or older 78 12 3 7 100 451 
Marital Status 
Never married 82 8 9 1 100 226 
Divorced, separated 92 3 3 2 100 114 
Widowed 74 12 6 8 100 221 
Married, no children present 85 9 4 2 100 1,016 
Married, children present 93 5 2 * 100 1,069 
Sex 
Male 88 7 4 1 100 1,239 
Female 87 8 3 2 100 1,412 
Relationship to Head 
of Household 
Head 87 8 4 1 100 1,356 
Wife 89 7 3 1 100 1,027 
Other relative 86 7 2 5 100 221 
Employment Status 
Employed 89 7 4 100 1,536 
Not employed 86 8 3 3 100 1,111 
Race | 
White 87 8 3 2 100 2,220 
Negro 91 5 4 * 100 428 
Total 87 8 4 1 100 2,651 


* Less than 0.5 per cent. 


cent of the designated respondents 60 years 
old or older were interviewed. The likeli- 
hood of a refusal increases markedly with 
age, in Detroit as elsewhere.*® But refusals 
are not the only reason for nonresponse from 
older persons. Physical and mental incapaci- 
ties, which occur most frequently among the 
aged, contribute to a lower response rate in 
this segment of the population. 

The high nonresponse rate among older 
adults contrasts sharply with the extreme 
ease with which these persons are contacted 
on the first call. This apparent contradiction 
emphasizes the conclusion that accessibility 
and a high response rate are not necessarily 
found together. Among all persons who 
eventually are interviewed, adults over 65 
years of age are decidedly more likely to 
grant an interview on the first call than are 
respondents of any other age; when all possi- 
ble respondents are considered, however, 


28 Lowe and McCormick, op. cit.; Stephan and 
McCarthy, op. cit., p. 262. 


older residents are also much more likely 
than are younger persons to refuse or to be 
incapable of granting an interview. The nega- 
tive influence of age on response rate is evi- 
denced in the low percentage of widowed 
adults who grant interviews. Only three out 
of every four widows or widowers in the 
original sample were interviewed. Both re- 
fusals and illness played important roles in 
the large number of nonresponses shown by 
this group. 

Single adults also tend to have rather low 
response rates. The nonresponse of unmar- 
ried Detroiters is not solely a result of re- 
fusals; one-tenth of the single adults in our 
sample were never contacted by an inter- 
viewer. Whether or not many of the desig- 
nated respondents who are impossible to 
find at home are simply refusing by avoiding 
the interviewer is unknown. 

Childless married adults have a somewhat 
lower response rate than do married adults 
with children. A disproportionate number 
of designated respondents with no minor 
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children in the home refuse to be interviewed. 
These persons are older, on the average, than 
adults with children, a fact which may 
explain at least some refusals in this group. 
The data indicate, however, that even when 
age differences are controlled, childless adults 
are slightly less likely to grant an interview 
than are married persons living with their 
children. Perhaps those parents who know 
that they must be home with the children 
whether or not they are interviewed, are 
therefore more willing than otherwise to 
sacrifice an hour of their time in the interests 
of social research. 

Sex, relationship to head of household, em- 
ployment status, and race are not highly as- 
sociated with differential response rates. Not- 
withstanding the relative inaccessibility of 
men, household heads, and workers, these 
persons do not contribute higher than aver- 
age nonresponse rates. Although gaining a 
completed interview with these respondents 
requires several call-backs, this fact does not 
mean that they are likely to refuse or to 
temain almost permanently inaccessible. 

The income, occupational status, and edu- 
cational levels of nonrespondents were not 
available. But an indication of the socio- 
economic status of nonrespondents who re- 
side in Detroit is provided through the use 
of data from the 1950 Census. Analysis of 
rental distributions for the 1955 Detroit 
Area Study’s survey reveals a slight under- 
representation of middle level rental units in 
the sample.”® In general, however, the me- 
dian rental of the blocks in which nonre- 
spondents are found is quite comparable to 
that of the city population as a whole and 
quite within allowable sampling error limits. 
The 1950 median rental of 1955 nonrespond- 
ents was 46 dollars; the comparable figure 
for the total city was 44 dollars. These data 
do not substantiate the rather common claim 
that upper socio-economic groups contribute 
a disproportionately large number of final 
nonrespondents.°®° 

Little evidence exists in the data to sup- 
port the proposition that refusals and not-at- 
homes tend to cancel one another.*! Although 


29 John Takeshita, “The Interview Response Rate 
for the 1954-55 Detroit Area Study.” The Detroit 
Area Study, 1955 (mimeographed). 

80 See, e.g., Parten, op. cit., pp. 412-413. 

31 Berelson, Lazarsfeld, and McPhee, loc. cit.; 
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refusals and age vary together, the propor- 
tion of not-at-homes is not related to age 
differences. Widowed adults have a high re- 
fusal rate and at least an average proportion 
of not-at-homes. Also, while unmarried De- 
troiters are very difficult to contact, almost 
as much nonresponse in this group is due 
to refusals as to not-at-homes. 


SAMPLE BIAS DUE TO NONRESPONSE 


Although nonresponse rates for a few 
segments of the population are rather high, 
this variation does not necessarily imply that 
strong biases exist in carefully planned prob- 
ability samples. Table 6 presents some sup- 
port for the claim that relatively small biases 
occur in a probability sample of the type 
described here when the over-all response 
rate is reasonably high. 

Seven characteristics of actual respondents 
and of all possible respondents in the original 
sample are compared in Table 6. Twenty-one 
specific comparisons are made and in only 
one instance does the bias in the completed 
sample, which derives from a failure to ob- 
tain a 100 per cent response rate, exceed two 
per cent.?* The sample overestimates the 
proportion of husband-wife-children family 
units by 2.5 percentage points. This and all 
other biases shown in Table 6 are in the 
directions which would be expected from 
earlier discussions. 

There is no claim here that possible sam- 
ple bias should be ignored. The objectives of 
a given research may conceivably dictate 
that a two percentage point error is unac- 
ceptable. In such instances the researcher 
may choose to incorporate the bias due to 
nonresponses into the total error of the 
survey estimates, using methods detailed in 
the literature.?? Or, he may choose to handle 


Stephan and McCarthy, op. cit., pp. 266-269; Helen 
Crossley and Raymond Fink, “Response and Non- 
Response in a Probability Sample,” International 
Journal of Opinion and Attitude Research, 5 (Spring, 
1951), pp. 1-20. 

32 Comparatively small biases are also reported 
for the Elmira study. Stephan and McCarthy, op. 
cit., pp. 266-269. Examples of surveys where large 
nonresponse rates resulted in dangerously large 
biases are found in Parten, op. cit., pp. 404-417. 

33 See, e.g., Z. W. Birnbaum and Monroe G. Sir- 
ken, “On the Total Error Due to Noninterview and 
to Random Sampling,” International Journal of 
Opinion Attitude Research, 4 (Summer, 1950), pp. 
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TABLE 6. RESPONDENTS AND TOTAL SAMPLE BY SELECTED CHARACTERISTICS 
—————————————— eeu ees 
eae Distribution Pevesniage 
Selected Characteristics Respondents Total Sample Point Difference 
Employment Status 
Employed 58.8 58.0 +0.8 
Not employed 41.2 42.0 —0.8 
Total 100.0 100.0 
Age 
21-34 years 36.4 34.6 +1.8 
35-59 years 48.1 48.4 —0.3 
60 years or older 15.5 17.0 —1.5 
Total 100.0 100.0 —~ 
Marital Status 
Never married 8.1 8.5 —0.4 
Divorced, separated 4.5 4.3 +0.2 
Widowed (pel 8.4 —1.3 
Married, no children present 37.4 38.4 —1.0 
Married, children present 42.9 40.4 +2.5 
Total 100.0 100.0 — 
Type of Dwelling Unit 
Apartment house or hotel 9.3 10.0 — 
Other multiple dwelling 25.1 25.0 +0.1 
Single family dwelling 65.6 65.0 +0.6 
Total 100.0 100.0 
Sex 
Male 47.0 46.7 +0.3 
Female 53.0 53.3 —0.3 
Total 100.0 100.0 
Relationship to Head of Household 
Head 50.9 51.2 —0.3 
Wife 39.3 38.7 +0.6 
Other relative 8.2 8.3 —0.1 
Nonrelative 1.6 1.8 —0.2 
Total 100.0 100.0 — 
Race 
White 83.0 83.8 —0.8 
Negro 17.0 16.2 +-0.8 
Total 100.0 100.0 
Number of cases 2,313 2,651 — 


this problem by adjusting his completed 
sample for nonresponse bias.** Nonetheless, 
for most surveys, biases of the magnitude 
reported here are quite reasonable and would 
generally have little influence on the final 
product of the analysis. 


CONCLUSION 


The ability of the student of human pop- 
ulations to draw a sample which is repre- 


179-191; William G. Cochran, Sampling Techniques, 
New York: Wiley, 1953, pp. 292-298. 

84 For clear descriptions and defenses of this pro- 
cedure, see Moser, op. cit., pp. 139-143 ; and Hansen, 
Hurwitz, and Madow, op. cit., pp. 56-59. 


sentative within known confidence limits of 
a given universe is well established. Even 
perfectly selected samples, however, can re- 
sult in very unrepresentative units of obser- 
vation if the designated sample varies sub- 
stantially from the final product. The prob- 
lems of varying accessibility among urban 
residents, differential response rates, and 
sample bias due to nonresponse have been 
discussed in some detail. The experiences 
reported here, we hope, will benefit future 
investigators who plan to employ survey 
techniques in the study of human popula- 
tions. 
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